Multi-input multi-output (MIMO) techniques are widely employed in transceivers to achieve high rate transmission in wireless communication system. In the downlink broadcast channels (BCs), the base station simultaneously transmits multiple signals to multiple mobile stations in the same frequency band. In the multi-user system that multiple cells coexist to form a wide coverage network, the desired signal of single user leads to interference to other users, thereby may be seriously degrading system performance. In current WiMAX and LTE standards, femtocells and cooperated macrocells are deployed to reduce transmission power and increase linking quality. However, due to the geographically irregular deployment, the interference arisen from the femtocells is even more serious than the interference arisen from the macrocells. To reduce the interference, multiple interference avoidance techniques are developed to reduce the transmission power and increasing overall capacity. However, the present interference avoidance techniques suffer from high complexity. Therefore, a practical interference avoidance technique with high performance is a major subject in the wireless communication field.